Segmentation from Motion: Combining Gabor- and Mallat-Wavelets to Overcome Aperture and Correspondence Problem

A segmentation-from-motion algorithm is presented, which is designed to be part of a general object recognition system. The key idea is to integrate information from Gabor-and Mallat-wavelet transform to overcome the aperture and the correspondence problem. The assumption is made that objects move fronto-parallel. Gabor-wavelet responses allow precise estimation of image ow vectors with low spatial resolution. A histogram over this image ow eld is evaluated, its local maxima providing motion hypotheses. These serve to reduce the correspondence problem on the Mallat-wavelet transform, which provides the required high resolution. The segmentation reliability is improved by integration over time. The system can segment several small, disconnected, and openworked objects, such as dot patterns. Several examples demonstrate the performance of the system and show, that the algorithm behaves reasonably even if the assumption of fronto-parallel motion is strongly violated.